Image processing system, x-ray diagnostic apparatus, and image processing method

ABSTRACT

An image processing system according to an embodiment includes a first aligning unit, an output unit, a second aligning unit, and a display unit. The first aligning unit aligns first three-dimensional medical image data with second three-dimensional medical image data. The output unit outputs, as output data, data obtained by adding alignment information to the first three-dimensional medical image data and to the second three-dimensional medical image data or synthetic data obtained by aligning and synthesizing the first three-dimensional medical image data with the second three-dimensional medical image data. The second aligning unit receives the output data and aligns the second three-dimensional medical image data with one or a plurality of pieces of X-ray image data. The display unit displays image data obtained by aligning the first three-dimensional medical image data with X-ray image data based on an alignment result.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/JP2013/075584, filed on Sep. 20, 2013 which claims the benefit ofpriority of the prior Japanese Patent Application No. 2012-207468, filedon Sep. 20, 2012 and Japanese Patent Application No. 2013-196006, filedon Sep. 20, 2013, the entire contents of which are incorporated hereinby reference.

FIELD

Embodiments described herein relate generally to an image processingsystem, an X-ray diagnostic apparatus, and an image processing method.

BACKGROUND

Conventionally, cardiac resynchronization therapy (CRT) is known as oneof methods for treating heart failure. The CRT is a treatment method forcorrecting asynchrony in heart motion and restoring a pump function ofthe heart to nearly a normal state by placing an electrode (a pacinglead) of a pacemaker into an area at which delay in propagation ofelectric stimulation occurs in the heart (hereinafter, referred to as adelay area). In the CRT, a doctor places the electrode into a veinclosest to the delay area while referring to an X-ray imagefluoroscopically captured by an X-ray diagnostic apparatus.

Conventionally, delay areas are diagnosed using information ofelectrophysiology (EP), or by EP mapping in recent years, for example.In recent years, it has been known that delay areas can be diagnosed bya non-invasive analysis using an ultrasonic diagnostic apparatus.Specifically, a method for analyzing heart wall motion quantitatively byechocardiography has been in practical use in recent years. Such ananalysis method can display an analysis image in which indices of localheart wall motion (e.g., strain) are mapped on an endocardium andbetween an endocardium and an epicardium in an ultrasound image in acolor tone varying depending on the value. Because a heart is a tissuein which a myocardium is moved by mechanical excitation caused byelectric stimulation, a delay area can be displayed as an area in whichthe heart wall motion is not synchronized (an asynchronous area) in theanalysis image. The CRT treatment, however, is carried out under X-rayfluoroscopic guidance, and the analysis image is simply notified to thedoctor as prior information to develop a treatment plan. Actually, it isnot yet realized that the doctor is informed of a position into whichthe pacing lead is to be placed under the X-ray fluoroscopic guidancefor the CRT treatment. On the other hand, there have been technologiesfor displaying an X-ray fluoroscopic image with another imagesuperimposed thereon being developed. Since an endocardial surface andan epicardial surface of a heart wall are hard to distinguish, it isdifficult to align an X-ray image with an analysis image, that is, anX-ray image with an ultrasound image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an exemplary configuration of an image processingsystem according to a first embodiment;

FIG. 2 is a block diagram of an exemplary configuration of an ultrasonicdiagnostic apparatus according to the first embodiment;

FIG. 3, FIG. 4, FIG. 5 and FIG. 6 are views for explaining an analyzingunit according to the first embodiment;

FIG. 7 is a view for explaining an aligning unit according to the firstembodiment;

FIG. 8 is a block diagram of an exemplary configuration of an X-raydiagnostic apparatus according to the first embodiment;

FIG. 9 is a diagram of processing units that carry out an imageprocessing method performed by the image processing system according tothe first embodiment;

FIG. 10 and FIG. 11 are views for explaining an example of processingperformed by the ultrasonic diagnostic apparatus according to the firstembodiment;

FIG. 12 and FIG. 13 are views for explaining an example of processingperformed by the X-ray diagnostic apparatus according to the firstembodiment;

FIG. 14 is a view of an example of image data displayed in the firstembodiment;

FIG. 15 is a flowchart for explaining an example of processing performedby the ultrasonic diagnostic apparatus according to the firstembodiment;

FIG. 16 is a flowchart for explaining an example of processing performedby the X-ray diagnostic apparatus according to the first embodiment;

FIG. 17 is a view for explaining a second embodiment; and

FIG. 18 is a flowchart for explaining an example of processing performedby an X-ray diagnostic apparatus according to the second embodiment.

DETAILED DESCRIPTION

An image processing system according to an embodiment includes a firstaligning unit, an output unit, a second aligning unit, and a displayunit. The first aligning unit aligns first three-dimensional medicalimage data with second three-dimensional medical image data, the firstthree-dimensional medical image data and the second three-dimensionalmedical image data being obtained by capturing a certain tissue of asubject. The output unit outputs, as output data, data obtained byadding alignment information to the first three-dimensional medicalimage data and to the second three-dimensional medical image data orsynthetic data obtained by aligning and synthesizing the firstthree-dimensional medical image data with the second three-dimensionalmedical image data. The second aligning unit receives the output dataand aligns the second three-dimensional medical image data with one or aplurality of pieces of X-ray image data, the X-ray image data beingobtained by capturing the certain tissue of the subject in one or aplurality of capturing directions, and corresponding to the respectivecapturing directions. The display unit displays image data obtained byaligning the first three-dimensional medical image data with the X-rayimage data of the certain tissue based on an alignment result of thefirst aligning unit and the second aligning unit.

Exemplary embodiments of an image processing system are described belowin greater detail with reference to the accompanying drawings.

First Embodiment

The following describes an exemplary configuration of an imageprocessing system according to a first embodiment. FIG. 1 is a diagramof an exemplary configuration of the image processing system accordingto the first embodiment.

As illustrated in FIG. 1, an image processing system 1 according to thefirst embodiment includes an ultrasonic diagnostic apparatus 100, anX-ray diagnostic apparatus 200, an X-ray computed tomography (CT)apparatus 300, an image storage device 400, and an image processingapparatus 500. The apparatuses illustrated in FIG. 1 are communicablewith one another directly or indirectly via an in-hospital local areanetwork (LAN) 600 installed in a hospital, for example. In the casewhere a picture archiving and communication system (PACS) is introducedinto a medical image diagnostic system, for example, the apparatusestransmit and receive medical images and the like to and from one anotherin conformity to the digital imaging and communications in medicine(DICOM).

By transmitting and receiving data conforming to the DICOM, each of theapparatuses illustrated in FIG. 1 can read and display data receivedfrom the other apparatuses. In the present embodiment, any dataconforming to an arbitrary standard may be transmitted and received aslong as each of the apparatuses can process the data received from theother apparatuses.

An operator adjusts the position of an ultrasound probe that performstwo-dimensional ultrasonic scanning, whereby the ultrasonic diagnosticapparatus 100 generates ultrasound image data on an arbitrary section.Furthermore, the ultrasonic diagnostic apparatus 100 performsthree-dimensional ultrasonic scanning with a mechanical 4D probe or a 2Darray probe, thereby generating three-dimensional ultrasound image data.The X-ray diagnostic apparatus 200 performs radiography with theposition of a C-arm supporting an X-ray tube and an X-ray detectorfixed, thereby generating two-dimensional X-ray image data. Theultrasonic diagnostic apparatus 100 and the X-ray diagnostic apparatus200 according to the first embodiment will be described later in detail.

The X-ray CT apparatus 300 includes a rotating frame that can rotate.The rotating frame supports an X-ray tube that irradiates a subject withan X-ray and an X-ray detector that detects the X-ray passing throughthe subject at positions facing each other. The X-ray CT apparatus 300rotates the rotating frame while irradiating the subject with an X-rayoutput from the X-ray tube, thereby acquiring data of the X-raysubjected to transmission, absorption, and attenuation in alldirections. The X-ray CT apparatus 300 reconstructs X-ray CT image datafrom the data thus acquired. The X-ray CT image data is a tomographicimage on a rotational plane (an axial plane) of the X-ray tube and theX-ray detector. In the X-ray detector, a plurality of arrays ofdetection elements, which are X-ray detection elements arrayed in achannel direction, are arranged along the body axis direction of thesubject. The X-ray CT apparatus 300 includes an X-ray detector in which16 arrays of detection elements are arranged, for example. In this case,the X-ray CT apparatus 300 reconstructs a plurality of (e.g., sixteen)X-ray CT image data along the body axis direction of the subject fromprojection data acquired by making one full rotation of the rotatingframe.

The X-ray CT apparatus 300 can reconstruct 500 pieces of X-ray CT imagedata including the entire heart as three-dimensional X-ray CT image databy helical scanning performed by moving a couchtop on which the subjectis placed while rotating the rotating frame, for example. Alternatively,the X-ray CT apparatus 300 includes an X-ray detector in which 320arrays of detection elements are arranged, for example. In this case,the X-ray CT apparatus 300 can reconstruct three-dimensional X-ray CTimage data all-inclusively covering the entire heart simply byperforming conventional scanning in which causes the rotating frame tomake one full rotation. By sequentially performing the helical scanningand the conventional scanning, the X-ray CT apparatus 300 can obtainthree-dimensional X-ray CT image data in a time series.

The first embodiment aligns the ultrasound image data obtained by theultrasonic diagnostic apparatus 100 with the X-ray image data obtainedby the X-ray diagnostic apparatus 200 using the three-dimensional X-rayCT image data. This will be described in detail after an explanation ismade of the entire configurations of the ultrasonic diagnostic apparatus100 and the X-ray diagnostic apparatus 200 according to the firstembodiment.

The image storage device 400 is a database that stores therein medicalimage data. Specifically, the image storage device 400 stores andretains medical image data transmitted from the ultrasonic diagnosticapparatus 100, the X-ray diagnostic apparatus 200, and the X-ray CTapparatus 300 in a storage unit of the image storage device 400. Themedical image data stored in the image storage device 400 is stored inassociation with accompanying information, such as a patient ID, anexamination ID, an apparatus ID, and a series ID.

The image processing apparatus 500 is a workstation and a personalcomputer (PC) used by doctors and laboratory technicians who work for ahospital to interpret a medical image, for example. An operator of theimage processing apparatus 500 performs a search using a patient ID, anexamination ID, an apparatus ID, a series ID, and other IDs, therebyacquiring necessary medical image data from the image storage device400. Alternatively, the image processing apparatus 500 may receive imagedata directly from the ultrasonic diagnostic apparatus 100, the X-raydiagnostic apparatus 200, and the X-ray CT apparatus 300. Besidesdisplaying a medical image for interpretation, the image processingapparatus 500 can perform various types of image processing on medicalimage data.

The following describes the case where the ultrasonic diagnosticapparatus 100 and the X-ray diagnostic apparatus 200 cooperate toperform an image processing method according to the present embodiment.A part or all of various types of processing performed by the ultrasonicdiagnostic apparatus 100 and the X-ray diagnostic apparatus 200 may beperformed by the X-ray CT apparatus 300 and the image processingapparatus 500.

The image processing system 1 is not necessarily applied to the casewhere the PACS is introduced. The image processing system 1 is alsoapplicable to the case where an electronic medical record system thatmanages electronic medical records accompanied with medical image datais introduced, for example. In this case, the image storage device 400is a database that stores therein the electronic medical records. Theimage processing system 1 is also applicable to the case where ahospital information system (HIS) or a radiology information system(RIS) is introduced, for example.

The following describes an exemplary configuration of the ultrasonicdiagnostic apparatus 100 illustrated in FIG. 1 with reference to FIG. 2.FIG. 2 is a block diagram of an exemplary configuration of theultrasonic diagnostic apparatus according to the first embodiment. Asillustrated in FIG. 1, the ultrasonic diagnostic apparatus 100 accordingto the first embodiment includes an ultrasound probe 110, a monitor 120,an input unit 130, an electrocardiograph 140, an apparatus main body150, a position sensor 160, and a transmitter 161.

The ultrasound probe 110 transmits and receives ultrasonic waves. Theultrasound probe 110 includes a plurality of piezoelectric transducerelements, for example. The plurality of piezoelectric transducerelements generate ultrasonic waves based on a driving signal suppliedfrom a transmitting and receiving unit 151 included in the apparatusmain body 150, which will be described later. The ultrasound probe 110receives reflected waves from a subject P and converts the reflectedwaves into an electrical signal. The ultrasound probe 110 furtherincludes a matching layer provided to the piezoelectric transducerelements and a backing member that prevents ultrasonic waves fromtraveling rearward from the piezoelectric transducer elements, forexample. The ultrasound probe 110 is connected to the apparatus mainbody 150 in an attachable and detachable manner.

If ultrasonic waves are transmitted from the ultrasound probe 110 to thesubject P, the ultrasonic waves thus transmitted are sequentiallyreflected at an acoustic impedance discontinuous surface in a bodytissue of the subject P. The ultrasonic waves are received by theplurality of piezoelectric transduce elements included in the ultrasoundprobe 110 as a reflected wave signal. The amplitude of the reflectedwave signal thus received depends on difference in the acousticimpedance on the discontinuous surface on which the ultrasonic waves arereflected. A reflected wave signal obtained when the ultrasonic pulsethus transmitted is reflected by a moving bloodstream, the surface of aheart wall, or the like depends on a velocity component of the movingobject with respect to an ultrasonic-wave transmitting direction becauseof the Doppler effect, thereby undergoing frequency shift.

The ultrasound probe 110 according to the first embodiment is anultrasound probe that can scan the subject P two-dimensionally and scanthe subject P three-dimensionally with ultrasonic waves. Specifically,the ultrasound probe 110 according to the first embodiment is amechanical 4D probe that scans the subject P two-dimensionally using theplurality of piezoelectric transduce elements arranged in a line andscans the subject P three-dimensionally by oscillating the plurality ofpiezoelectric transduce elements at a predetermined angle (anoscillation angle). Alternatively, the ultrasound probe 110 according tothe first embodiment is a 2D array probe that can perform ultrasonicscanning on the subject P three-dimensionally with the plurality ofpiezoelectric transduce elements arranged in a matrix. The 2D arrayprobe can also scan the subject P two-dimensionally by focusing andtransmitting the ultrasonic waves.

The input unit 130 includes a mouse, a keyboard, a button, a panelswitch, a touch command screen, a foot switch, a trackball, and ajoystick, for example. The input unit 130 receives various types ofsetting requests from an operator of the ultrasonic diagnostic apparatus100 and transfers the various types of setting requests thus received tothe apparatus main body 150.

The monitor 120 displays a graphical user interface (GUI) through whichthe operator of the ultrasonic diagnostic apparatus 100 inputs varioustypes of setting request with the input unit 130 and displays ultrasoundimage data generated in the apparatus main body 150, for example.

The electrocardiograph 140 acquires an electrocardiogram (ECG) of thesubject P as a biomedical signal of the subject P. Theelectrocardiograph 140 transmits the ECG thus acquired to the apparatusmain body 150.

The position sensor 160 and the transmitter 161 are devices that acquirethe position information of the ultrasound probe 110. The positionsensor 160 is a magnetic sensor attached to the ultrasound probe 110,for example. The transmitter 161 is arranged at an arbitrary positionand generates a magnetic field extending outward from the transmitter161 with the transmitter 161 being the center.

The position sensor 160 detects the three-dimensional magnetic fieldgenerated by the transmitter 161. The position sensor 160 derives theposition (coordinates and an angle) of the position sensor 160 in aspace with its origin at the transmitter 161 based on the information ofthe magnetic field thus detected and transmits the position thus derivedto the apparatus main body 150. The position sensor 160 transmits thethree-dimensional coordinates and angle at which the position sensor 160is positioned to the apparatus main body 150 as the three-dimensionalposition information of the ultrasound probe 110.

The present embodiment is also applicable to the case where the positioninformation of the ultrasound probe 110 is acquired by a system otherthan the position detecting system using the position sensor 160 and thetransmitter 161. The present embodiment may acquire the positioninformation of the ultrasound probe 110 using a gyro sensor and anacceleration sensor, for example.

The apparatus main body 150 is a device that generates ultrasound imagedata based on a reflected wave signal received by the ultrasound probe110. The apparatus main body 150 illustrated in FIG. 2 is a device thatcan generate two-dimensional ultrasound image data based ontwo-dimensional reflected wave data received by the ultrasound probe110. Furthermore, the apparatus main body 150 illustrated in FIG. 1 is adevice that can generate three-dimensional ultrasound image data basedon three-dimensional reflected wave data received by the ultrasoundprobe 110.

As illustrated in FIG. 2, the apparatus main body 150 includes thetransmitting and receiving unit 151, a B-mode processing unit 152, aDoppler processing unit 153, an image generating unit 154, an imagememory 155, an image processing unit 156, a control unit 157, aninternal storage unit 158, and an interface unit 159.

The transmitting and receiving unit 151 includes a pulse generator, atransmission delay unit, a pulser, and other components, and supplies adriving signal to the ultrasound probe 110. The pulse generatorrepeatedly generates a rate pulse that forms transmission ultrasonicwaves at a predetermined rate frequency. The transmission delay unitsupplies delay times required for the respective piezoelectrictransducer elements to focus ultrasonic waves generated from theultrasound probe 110 into a beam and to determine the transmissiondirectivity to the respective rate pulses generated by the pulsegenerator. The pulser applies a driving signal (a driving pulse) to theultrasound probe 110 at a timing based on the rate pulse. Specifically,the transmission delay unit changes the delay times supplied to therespective rate pulses, thereby arbitrarily controlling the direction oftransmission of the ultrasonic waves transmitted from the piezoelectrictransducer element surface.

The transmitting and receiving unit 151 has a function toinstantaneously change a transmission frequency, a transmission drivingvoltage, and other elements so as to perform a predetermined scanningsequence based on an instruction issued from the control unit 157, whichwill be described later. Specifically, the transmission driving voltageis changed by a linear-amplifier oscillating circuit that caninstantaneously change the value of the transmission driving voltage ora mechanism that electrically switches a plurality of power-supplyunits.

The transmitting and receiving unit 151 further includes apre-amplifier, an analog/digital (A/D) converter, a reception delayunit, and an adder, for example. The transmitting and receiving unit 151performs various types of processing on a reflected wave signal receivedby the ultrasound probe 110, thereby generating reflected wave data. Thepre-amplifier amplifies a reflected wave signal on each channel. The A/Dconverter performs A/D conversion on the reflected wave signal thusamplified. The reception delay unit supplies a delay time required todetermine the reception directivity. The adder performs addition on thereflected wave signal processed by the reception delay unit, therebygenerating reflected wave data. The addition performed by the adderemphasizes a reflection component in a direction corresponding to thereception directivity of the reflected wave signal. Based on thereception directivity and the transmission directivity, a synthetic beamfor transmitting and receiving ultrasonic waves is formed.

To scan the subject P two-dimensionally, the transmitting and receivingunit 151 causes the ultrasound probe 110 to transmit a two-dimensionalultrasonic beam. The transmitting and receiving unit 151 generatestwo-dimensional reflected wave data from a two-dimensional reflectedwave signal received by the ultrasound probe 110. To scan the subject Pthree-dimensionally, the transmitting and receiving unit 151 causes theultrasound probe 110 to transmit a three-dimensional ultrasonic beam.The transmitting and receiving unit 151 generates three-dimensionalreflected wave data from a three-dimensional reflected wave signalreceived by the ultrasound probe 110.

An output signal from the transmitting and receiving unit 151 may havevarious forms being selectable, including a signal containing phaseinformation, which is called a radio frequency (RF) signal, andamplitude information obtained after envelope detection is performed,for example.

The B-mode processing unit 152 receives reflected wave data from thetransmitting and receiving unit 151. The B-mode processing unit 152performs logarithmic amplification, envelope detection, and otherprocessing on the reflected wave data, thereby generating data (B-modedata) representing the signal intensity by the intensity of brightness.

The Doppler processing unit 153 performs a frequency analysis onvelocity information in the reflected wave data received from thetransmitting and receiving unit 151. The Doppler processing unit 153extracts a bloodstream, a tissue, and a contrast medium echo componentby the Doppler effect and generates data (Doppler data) by extractingmoving object information, such as velocity, dispersion, and power, atmultiple points.

The B-mode processing unit 152 and the Doppler processing unit 153according to the first embodiment can process both two-dimensionalreflected wave data and three-dimensional reflected wave data. In otherwords, the B-mode processing unit 152 generates two-dimensional B-modedata from two-dimensional reflected wave data and generatesthree-dimensional B-mode data from three-dimensional reflected wavedata. The Doppler processing unit 153 generates two-dimensional Dopplerdata from two-dimensional reflected wave data and generatesthree-dimensional Doppler data from three-dimensional reflected wavedata.

The image generating unit 154 generates ultrasound image data from thedata generated by the B-mode processing unit 152 and the Dopplerprocessing unit 153. In other words, the image generating unit 154generates two-dimensional B-mode image data representing the intensityof reflected waves by the brightness from the two-dimensional B-modedata generated by the B-mode processing unit 152. Furthermore, the imagegenerating unit 154 generates two-dimensional Doppler image dataindicating the moving object information from the two-dimensionalDoppler data generated by the Doppler processing unit 153. Thetwo-dimensional Doppler image data is a velocity image, a dispersionimage, a power image, or a combination image of these images.

Typically, the image generating unit 154 converts (scan-converts) ascanning-line signal row in ultrasonic scanning into a scanning-linesignal row in a video format as exemplified by television and the like,thereby generating ultrasound image data for display. Specifically, theimage generating unit 154 performs coordinate transformation based onthe form of the ultrasonic scanning performed by the ultrasound probe110, thereby generating the ultrasound image data for display. The imagegenerating unit 154 performs various types of image processing besidesthe scan-conversion on a plurality of image frames thus scan-converted.Examples of the various types of image processing include imageprocessing for regenerating a brightness average value image (smoothingprocessing) and image processing using a differential filter in an image(edge emphasizing processing). The image generating unit 154 synthesizescharacter information of various types of parameters, a scale, and abody mark on the ultrasound image data, for example.

In other words, the B-mode data and the Doppler data are ultrasoundimage data before the scan-conversion. The data generated by the imagegenerating unit 154 is ultrasound image data for display after thescan-conversion. The B-mode data and the Doppler data are also referredto as raw data.

The image generating unit 154 performs coordinate transformation on thethree-dimensional B-mode data generated by the B-mode processing unit152, thereby generating three-dimensional B-mode image data.Furthermore, the image generating unit 154 performs coordinatetransformation on the three-dimensional Doppler data generated by theDoppler processing unit 153, thereby generating three-dimensionalDoppler image data. In other words, the image generating unit 154generates “three-dimensional B-mode image data and three-dimensionalDoppler image data” as “three-dimensional ultrasound image data”.

To generate various types of two-dimensional image data for displayingthree-dimensional ultrasound image data (volume data) on the monitor120, the image generating unit 154 performs rendering on the volumedata. Examples of the rendering performed by the image generating unit154 include processing for generating multi-planer reconstruction (MPR)image data from volume data by performing MPR. Examples of the renderingperformed by the image generating unit 154 further include processingfor performing “curved MPR” on volume data and processing for performing“maximum intensity projection” on volume data. Examples of the renderingperformed by the image generating unit 154 further include volumerendering (VR) for generating two-dimensional image data reflectingthree-dimensional information.

The image memory 155 is a memory that stores therein image data fordisplay generated by the image generating unit 154. The image memory 155can also store therein data generated by the B-mode processing unit 152and the Doppler processing unit 153. The B-mode data and the Dopplerdata stored in the image memory 155 can be retrieved by the operatorafter a diagnosis, for example, and are converted into ultrasound imagedata for display via the image generating unit 154. The image generatingunit 154 stores ultrasound image data and a time of ultrasonic scanningperformed to generate the ultrasound image data in the image memory 155in association with an ECG transmitted from the electrocardiograph 140.An analyzing unit 156 a and the control unit 157, which will bedescribed later, refer to the data stored in the image memory 155,thereby acquiring a cardiac time phase in the ultrasonic scanningperformed to generate the ultrasound image data.

The internal storage unit 158 stores therein various types of data, suchas a control program for performing transmission and reception ofultrasonic waves, image processing, and display processing, diagnosisinformation (e.g., a patient ID and findings of a doctor), a diagnosisprotocol, and various types of body marks. Furthermore, the internalstorage unit 158 is used to retain image data stored in the image memory155 as needed, for example. The data stored in the internal storage unit158 can be transferred to external apparatuses via the interface unit159, which will be described later. Data stored in the externalapparatuses can be transferred to the internal storage unit 158 via theinterface unit 159, which will be described later. The externalapparatuses correspond to the X-ray diagnostic apparatus 200, the X-rayCT apparatus 300, the image storage device 400, and the image processingapparatus 500 illustrated in FIG. 1, for example.

The image processing unit 156 is provided to the apparatus main body 150to perform a computer-aided diagnosis (CAD). The image processing unit156 acquires ultrasound image data stored in the image memory 155 andperforms an image analysis thereon. The image processing unit 156 storesthe analysis result in the image memory 155 and the internal storageunit 158.

As illustrated in FIG. 2, the image processing unit 156 includes theanalyzing unit 156 a and an aligning unit 156 b. The analyzing unit 156a analyzes a time-series three-dimensional ultrasound image data groupgenerated by performing three-dimensional ultrasonic scanning on thesubject P, thereby generating three-dimensional analysis image datarelating to local motion in a certain tissue.

The certain tissue corresponds to a heart, and the analyzing unit 156 agenerates information relating to motion in each region of the heartwall. The analyzing unit 156 a generates analysis image data in whichheart wall motion information is mapped on the endocardium and betweenthe endocardium and the epicardium in the ultrasound image data. Theanalyzing unit 156 a according to the first embodiment uses thethree-dimensional ultrasound image data group, thereby generatingthree-dimensional time-series data of the heart wall motion information.

The following describes the analysis performed by the analyzing unit 156a according to the first embodiment with reference to FIG. 3 to FIG. 6.FIG. 3 to FIG. 6 are views for explaining the analyzing unit accordingto the first embodiment.

The operator uses the ultrasound probe 110 that can performthree-dimensional scanning, thereby performing three-dimensionalscanning on the left side of the heart of the subject P by apicalapproach for a time period equal to or longer than one heartbeat, forexample. As a result, the image generating unit 154 generates aplurality of pieces of time-series three-dimensional ultrasound imagedata in a time period equal to or longer than one heartbeat and storesthe plurality of pieces of three-dimensional ultrasound image data inthe image memory 155. The plurality of pieces of three-dimensionalultrasound image data stored in the image memory 155 are athree-dimensional ultrasound image data group generated by performingultrasonic scanning on the heart including at least the left ventriclefor a time period equal to or longer than one heartbeat. Thethree-dimensional ultrasound image data group is a three-dimensionalB-mode image data group.

As illustrated in FIG. 3, the analyzing unit 156 a acquires theplurality of pieces of time-series three-dimensional ultrasound imagedata in a time period equal to or longer than one heartbeat. Each of theplurality of pieces of three-dimensional ultrasound image data includesthe left ventricle of the subject P.

The analyzing unit 156 a derives time-series data of the heart wallmotion information in the left ventricle from the three-dimensionalultrasound image data group. Specifically, the analyzing unit 156 a usesthe result of tracking of tracking points described below performed byprocessing including pattern matching between a plurality of pieces ofimage data, thereby deriving the heart wall motion information. Morespecifically, the analyzing unit 156 a uses the result of 3D speckletracking (hereinafter, referred to as “3DT”) performed onthree-dimensional moving image data obtained by a three-dimensionalechocardiography method, thereby deriving the heart wall motioninformation. The speckle tracking method is a method for estimatingaccurate motion by performing the pattern matching in combination withan optical flow method and various types of spatiotemporalinterpolation, for example. Some speckle tracking methods estimate themotion without performing the pattern matching.

The input unit 130, for example, receives a display request of the firstframe (first volume) of the three-dimensional ultrasound image datagroup from the operator. The control unit 157 to which the displayrequest is transferred reads three-dimensional ultrasound image data ofthe first frame from the image memory 155 and displays thethree-dimensional ultrasound image data on the monitor 120. The controlunit 157, for example, causes the image generating unit 154 to generatea plurality of pieces of MPR image data by cutting the three-dimensionalultrasound image data of the first frame on sections in a plurality ofdirections and to display the plurality of pieces of MPR image data onthe monitor 120. The monitor 120 displays the plurality of pieces of MPRimage data as illustrated in FIG. 4, for example.

In the example of FIG. 4, the monitor 120 displays MPR image data of aplane A in an area A. In the example of FIG. 4, the monitor 120 displaysMPR image data of a plane B in an area B. In the example of FIG. 4, themonitor 120 displays MPR image data of a plane C of a C3 level closer tothe cardiac apex in an area C3. In the example of FIG. 4, the monitor120 displays MPR image data of the plane C of a C7 level closer to thecardiac base in the area C7. In the example of FIG. 4, the monitor 120displays MPR image data of the plane C of a C5 level intermediatebetween the cardiac apex and the cardiac base in an area C5. In theexample of FIG. 4, the area C3, the area C5, and an area C7 are arrangedin descending order in the left display area. The area A is arranged onthe right side of the area C3 and the area C5, and the area B isarranged on the right side of the area A.

In the example of FIG. 4, the monitor 120 displays a volume renderingimage of the three-dimensional ultrasound image data of the first frameand the ECG in the lower right area of the screen.

The operator refers to the plurality of pieces of MPR image datadisplayed on the monitor 120, thereby setting a plurality of trackingpoints used for performing 3DT. The operator, for example, traces thepositions of the endocardium of the left ventricle and the epicardium ineach piece of MPR image data, thereby specifying the endocardial outlineand the epicardial outline. The analyzing unit 156 a forms athree-dimensional endocardial outline and a three-dimensional epicardialoutline from the endocardial outline and the epicardial outline thusspecified. The analyzing unit 156 a sets the points forming thethree-dimensional endocardial outline in the first frame as trackingpoints as illustrated in FIG. 5. The analyzing unit 156 a also sets thepoints forming the three-dimensional epicardial outline in the firstframe as tracking points, which is not illustrated. The analyzing unit156 a sets template data to each of the plurality of tracking points setin the first frame. The template data is formed of a plurality of voxelsaround each tacking point.

The analyzing unit 156 a searches for an area that matches best to thespeckle pattern of the template data between two frames, therebytracking the template data to find out to which position the templatedata moves in the next frame. Thus, the analyzing unit 156 a tracks eachof the tracking points in the first frame to find out to which positionsin the n-th frame the tracking points in the first frame move asillustrated in FIG. 5. The mesh used for setting the tracking points maybe set by the analyzing unit 156 a detecting the endocardial surface andthe epicardial surface of the left ventricle included in the firstframe.

The analyzing unit 156 a performs 3DT on the three-dimensionalultrasound image data group for the entire left ventricle (e.g., theendocardium of the left ventricle and the epicardium of the leftventricle). Based on the result of the 3DT performed on thethree-dimensional ultrasound image data group, the analyzing unit 156 agenerates time-series data of the heat wall motion information on eachtracking point. The analyzing unit 156 a derives strain as the heartwall motion information from the result of the 3DT of the endocardiumand the epicardium, for example. The analyzing unit 156 a derives strainin the longitudinal direction (LS), strain in the circumferentialdirection (CS), and strain in the radial direction (RS).

Alternatively, the analyzing unit 156 a derives the area change ratio(AC) of the endocardial surface of the left ventricle as the heart wallmotion information from the result of the 3DT of the endocardium, forexample. Still alternatively, the analyzing unit 156 a may derivedisplacement from the result of the 3DT of the endocardium or theepicardium, for example. In the case where the displacement is employedas the heart wall motion information, the analyzing unit 156 a canderive displacement in the longitudinal direction (LD) and displacementin the radial direction (RD). Alternatively, the analyzing unit 156 amay derive absolute displacement (AD) of the tracking points at a timephase other than a reference phase with respect to the positions of therespective tracking points at a reference time phase (e.g., an R-wave).Still alternatively, to grasp the asynchrony in the motion of the heart,the analyzing unit 156 a may derive an analysis result of mapping oftime when the strain value is equal to or larger than a certain value oran analysis result of mapping of time when the strain value reaches themaximum.

The analyzing unit 156 a may generate the time-series data of the heartwall motion information for each tracking point or for each localregion. The analyzing unit 156 a derives local heart wall motioninformation using a segmented region of 16 or 17 segments, which isrecommended by the American Society of Echocardiography (ASE) and theAmerican Heart Association (AHA), for example. Examples of the segmentsrecommended by the ASE include an anterior wall septum (ant-sept.), ananterior wall (ant.), a lateral wall (lat.), a posterior wall (post.),an inferior wall (inf.), and a septum (sept.)

The analyzing unit 156 a converts the values of the heart wall motioninformation obtained at the respective tracking points into color scalesand maps the values onto a surface rendering image of thethree-dimensional endocardial outline, thereby generatingthree-dimensional analysis image data as illustrated in FIG. 6, forexample. The operator can observe the three-dimensional analysis imagedata illustrated in FIG. 6 on the monitor 120 in various directions bymoving the viewpoint position. Alternatively, the analyzing unit 156 aconverts the values of the heart wall motion information obtained at therespective tracking points into color scales and maps the values onto apolar-map of 16 segments, thereby generating three-dimensional analysisimage data, for example.

Referring back to FIG. 2, the aligning unit 156 b aligns ultrasoundimage data with a second type of three-dimensional medical image data.The second type of three-dimensional medical image data corresponds tothree-dimensional X-ray CT image data received from the X-ray CTapparatus 300, for example. Alternatively, the second type ofthree-dimensional medical image data corresponds to three-dimensionalMRI image data received from a magnetic resonance imaging (MRI)apparatus, which is not illustrated in FIG. 1, for example. Theultrasonic diagnostic apparatus 100 according to the first embodimentcan cause the image generating unit 154 to generate medical image dataon nearly the same section as the section of two-dimensional ultrasonicscanning performed to generate two-dimensional ultrasound image data byprocessing of the position sensor 160 and the aligning unit 156 b anddisplays the medical image data on the monitor 120.

Before performing echocardiography of the subject P with the ultrasoundprobe 110, for example, the operator requests transmission ofthree-dimensional X-ray CT image data obtained by capturing the heart ofthe subject P. Furthermore, the operator adjusts the position of thesection for MPR processing via the input unit 130 such thattwo-dimensional X-ray CT image data depicting the examination area ofthe subject P is displayed on the monitor 120.

Under the control of the aligning unit 156 b, the image generating unit154 generates two-dimensional X-ray CT image data obtained by cuttingthe three-dimensional X-ray CT image data on the section (hereinafter,referred to as an initial section) adjusted by the operator. The monitor120 displays the two-dimensional X-ray CT image data generated by theimage generating unit 154. The operator operates the ultrasound probe110 so as to perform ultrasonic scanning on the same section as that ofthe X-ray CT image data displayed on the monitor 120. If it isdetermined that the sections of the two-dimensional X-ray CT image dataand the two-dimensional ultrasound image data displayed on the monitor120 are nearly the same, the operator specifies three correspondingpoints on both image data, for example. Alternatively, the operatorspecifies one or more corresponding points and axes (lines) on bothimage data, for example. The operator then presses a confirmation buttonof the input unit 130. The aligning unit 156 b sets three-dimensionalposition information of the ultrasound probe 110 acquired from theposition sensor 160 when the confirmation button is pressed as initialposition information. The aligning unit 156 b aligns the coordinatesystem of the two-dimensional ultrasound image data with the coordinatesystem of the three-dimensional X-ray CT image data using the points orthe lines corresponding to each other. FIG. 7 is a view for explainingthe aligning unit according to the first embodiment.

Subsequently, the aligning unit 156 b acquires the three-dimensionalposition information of the ultrasound probe 110 when generatingtwo-dimensional ultrasound image data B illustrated in FIG. 7 from theposition detecting system formed of the position sensor 160 and thetransmitter 161. The aligning unit 156 b acquires movement informationbetween the three-dimensional position information thus acquired and theinitial position information. Based on the movement information thusacquired, the aligning unit 156 b changes the position of the initialsection, thereby resetting the section for MPR. Under the control of thealigning unit 156 b, the image generating unit 154 generatestwo-dimensional X-ray CT image data C from three-dimensional X-ray CTimage data A illustrated in FIG. 7 on the section reset by the aligningunit 156 b. Under the control of the aligning unit 156 b, the monitor120 displays the two-dimensional X-ray CT image data C and thetwo-dimensional ultrasound image data B in parallel as illustrated inFIG. 7. The explanation has been made of the case where alignment isperformed using the position sensor 160. Alignment of thethree-dimensional ultrasound image data with the three-dimensional X-rayCT image data (or three-dimensional MRI image data), however, can beperformed as well without using the position sensor 160, by specifyingthree or more common feature points on both image data after acquiringthe three-dimensional ultrasound image data. For example, if displayingboth MPR image data, setting the common features individually,synchronizing the images at a time when more than three-points have beenset, then a similar concurrent display as using the position sensor 160by an interface such as a mouse, becomes possible.

The concurrent display function enables the operator to simultaneouslyobserve an ultrasound image and an X-ray CT image on nearly the samesection as that of the ultrasound image, for example. By performingtwo-dimensional scanning with the ultrasound probe 110 that can performthree-dimensional scanning and by acquiring the initial positioninformation and the position information of the points and the lines incorrespondence, the aligning unit 156 b can identify nearly the samethree-dimensional region as that on which three-dimensional ultrasonicscanning is performed in the three-dimensional X-ray CT image data. Thealigning unit 156 b can also align voxels constituting thethree-dimensional ultrasound image data with respective voxelsconstituting the three-dimensional X-ray CT image data.

In other words, the aligning unit 156 b can align three-dimensionalultrasound image data with three-dimensional X-ray CT image data andalign three-dimensional ultrasound image data with three-dimensional MRIimage data. Furthermore, the aligning unit 156 b can alignthree-dimensional analysis image data with three-dimensional X-ray CTimage data using alignment information of the three-dimensionalultrasound image data and the three-dimensional X-ray CT image data.Similarly, the aligning unit 156 b can align three-dimensional analysisimage data with three-dimensional MRI image data using alignmentinformation of the three-dimensional ultrasound image data and thethree-dimensional MRI image data. In the case where thethree-dimensional X-ray CT image data and the three-dimensional MRIimage data are three-dimensional contrast enhanced image data obtainedby contrast enhanced radiography, the aligning unit 156 b can alignthree-dimensional contrast enhanced region data segmented from thethree-dimensional contrast enhanced image data with three-dimensionalanalysis image data.

Referring back to FIG. 2, the control unit 157 controls the entireprocessing of the ultrasonic diagnostic apparatus 100. Specifically, thecontrol unit 157 controls the processing of the transmitting andreceiving unit 151, the B-mode processing unit 152, the Dopplerprocessing unit 153, the image generating unit 154, and the analyzingunit 156 a based on various types of setting requests received from theoperator via the input unit 130 and various types of control programsand various types of data read from the internal storage unit 158. Thecontrol unit 157 performs control such that the monitor 120 displaysultrasound image data for display stored in the image memory 155 and theinternal storage unit 158. Furthermore, the control unit 157 performscontrol such that the monitor 120 displays the processing result of theanalyzing unit 156 a.

The control unit 157 outputs the processing result of the analyzing unit156 a and other data to the external apparatuses via the interface unit159, which will be described later. The external apparatuses correspondto the X-ray diagnostic apparatus 200, the X-ray CT apparatus 300, theimage storage device 400, and the image processing apparatus 500illustrated in FIG. 1, for example. The control unit 157 according tothe first embodiment includes an output unit 157 a illustrated in FIG. 2serving as a processing unit that outputs output data and controls thedata format of the output data. The processing performed by the outputunit 157 a will be described later in detail.

The interface unit 159 is an interface provided for the input unit 130,an in-hospital LAN 600, the X-ray diagnostic apparatus 200, the X-ray CTapparatus 300, the image storage device 400, and the image processingapparatus 500. Various types of setting information and various types ofinstructions received from the operator by the input unit 130 aretransferred to the control unit 157 by the interface unit 159, forexample. Output data output from the output unit 157 a is transmitted tothe X-ray diagnostic apparatus 200 by the interface unit 159 via thein-hospital LAN 600, for example. Data including three-dimensionalmedical image data transmitted from the X-ray CT apparatus 300 and theimage storage device 400 is stored in the internal storage unit 158 viathe interface unit 159, for example.

The following describes an exemplary configuration of the X-raydiagnostic apparatus 200 illustrated in FIG. 1 with reference to FIG. 8.FIG. 8 is a block diagram of an exemplary configuration of the X-raydiagnostic apparatus according to the first embodiment. As illustratedin FIG. 8, the X-ray diagnostic apparatus 200 according to the firstembodiment includes an X-ray high-voltage generator 211, an X-ray tube212, an X-ray aperture device 213, a couchtop 214, a C-arm 215, and anX-ray detector 216. The X-ray diagnostic apparatus 200 according to thefirst embodiment further includes a C-arm rotation and movementmechanism 217, a couchtop movement mechanism 218, a C-arm and couchtopmechanism control unit 219, an aperture control unit 220, a systemcontrol unit 221, an input unit 222, and a display unit 223. The X-raydiagnostic apparatus 200 according to the first embodiment furtherincludes an image data generating unit 224, an image data storage unit225, and an image processing unit 226.

The X-ray high-voltage generator 211 generates a high voltage andsupplies the high voltage thus generated to the X-ray tube 212 under thecontrol of the system control unit 221. The X-ray tube 212 uses the highvoltage supplied from the X-ray high-voltage generator 211, therebygenerating an X-ray.

The X-ray aperture device 213 limits the X-ray generated by the X-raytube 212 such that a region of interest of the subject P is selectivelyirradiated with the X-ray under the control of the aperture control unit220. The X-ray aperture device 213 includes four slidable apertureblades, for example. Under the control of the aperture control unit 220,the X-ray aperture device 213 slides the aperture blades to limit theX-ray generated by the X-ray tube 212, thereby irradiating the subject Pwith the X-ray. The couchtop 214 is a bed on which the subject P isplaced and is arranged on a couch, which is not illustrated.

The X-ray detector 216 detects an X-ray transmitting through the subjectP. The X-ray detector 216 includes detection elements arrayed in amatrix, for example. The detection elements each convert the X-raytransmitting through the subject P into an electrical signal, accumulatethe electrical signal, and transmit the electrical signal thusaccumulated to the image data generating unit 224.

The C-arm 215 holds the X-ray tube 212, the X-ray aperture device 213,and the X-ray detector 216. The X-ray tube 212 and the X-ray aperturedevice 213 are arranged in a manner facing the X-ray detector 216 withthe subject P interposed therebetween by the C-arm 215.

The C-arm rotation and movement mechanism 217 is a mechanism thatrotates and moves the C-arm 215. The couchtop movement mechanism 218 isa mechanism that moves the couchtop 214. The C-arm and couchtopmechanism control unit 219 controls the C-arm rotation and movementmechanism 217 and the couchtop movement mechanism 218 under the controlof the system control unit 221, thereby adjusting rotation and movementof the C-arm 215 and movement of the couchtop 214. The aperture controlunit 220 adjusts the degree of opening of the aperture blades includedin the X-ray aperture device 213 under the control of the system controlunit 221, thereby controlling the irradiation range of the X-ray withwhich the subject P is irradiated.

The image data generating unit 224 uses the electrical signal convertedfrom the X-ray by the X-ray detector 216, thereby generating X-ray imagedata. The image data generating unit 224 then stores the X-ray imagedata thus generated in the image data storage unit 225. The image datagenerating unit 224, for example, performs current/voltage conversion,analog/digital (A/D) conversion, and parallel/serial conversion on theelectrical signal received from the X-ray detector 216, therebygenerating X-ray image data.

The image data storage unit 225 stores therein image data generated bythe image data generating unit 224. The image processing unit 226performs various types of image processing on the image data stored inthe image data storage unit 225. The image processing performed by theimage processing unit 226 will be described later in detail.

The input unit 222 receives various types of instructions issued from anoperator, such as a doctor and a technician, who operates the X-raydiagnostic apparatus 200. The input unit 222 includes a mouse, akeyboard, a button, a trackball, and a joystick, for example. The inputunit 222 transfers the instructions received from the operator to thesystem control unit 221.

The display unit 223 displays a GUI that receives an instruction fromthe operator and the image data stored in the image data storage unit225, for example. The display unit 223 includes a monitor, for example.The display unit 223 may include a plurality of monitors.

The system control unit 221 controls the entire operation of the X-raydiagnostic apparatus 200. The system control unit 221, for example,controls the X-ray high-voltage generator 211 based on an instructionfrom the operator transferred from the input unit 222, thereby adjustingthe voltage supplied to the X-ray tube 212. Thus, the system controlunit 221 controls the amount of the X-ray with which the subject P isirradiated and ON/OFF of irradiation of the X-ray. The system controlunit 221, for example, controls the C-arm and couchtop mechanism controlunit 219 based on an instruction from the operator, thereby adjustingrotation and movement of the C-arm 215 and movement of the couchtop 214.The system control unit 221, for example, controls the aperture controlunit 220 based on an instruction from the operator, thereby adjustingthe degree of opening of the aperture blades included in the X-rayaperture device 213. Thus, the system control unit 221 controls theirradiation range of the X-ray with which the subject P is irradiated.

The system control unit 221 controls the image data generationprocessing performed by the image data generating unit 224, the imageprocessing performed by the image processing unit 226, and otherprocessing based on an instruction from the operator. The system controlunit 221 performs control such that the monitor of the display unit 223displays the GUI that receives an instruction from the operator anddisplays the image data stored in the image data storage unit 225, forexample.

To perform the various types of processing using output data receivedfrom the ultrasonic diagnostic apparatus 100, the system control unit221 includes an acquiring unit 221 a as illustrated in FIG. 3. Theacquiring unit 221 a is a processing unit that performs alignment, whichwill be described later, for example. In other words, assuming that thealigning unit 156 b corresponds to a first aligning unit, the imageprocessing system 1 includes the acquiring unit 221 a corresponding to asecond aligning unit. The processing performed by the acquiring unit 221a will be described later in detail.

An interface unit 227 is an interface provided for the in-hospital LAN600, the ultrasonic diagnostic apparatus 200, the X-ray CT apparatus300, the image storage device 400, and the image processing apparatus500. The interface unit 227 according to the present embodiment, forexample, receives output data output from the ultrasonic diagnosticapparatus 100 and transfers the output data thus received to theacquiring unit 221 a included in the system control unit 221.

The explanation has been made of the entire configuration of the imageprocessing system 1 according to the first embodiment. With thisconfiguration, the image processing system 1 according to the firstembodiment specifies an area requiring treatment by ultrasonographyusing the ultrasonic diagnostic apparatus 100. Specifically, anasynchronous area into which an electrode of a pacemaker is to be placedis specified from the three-dimensional analysis image data generated bythe analyzing unit 156 a in cardiac resynchronization therapy (CRT). Inthe CRT, the doctor places the electrode into a vein closest to theasynchronous area while refereeing to an X-ray image fluoroscopicallycaptured by the X-ray diagnostic apparatus 200. Because the endocardialsurface and the epicardial surface of the heart wall are hard todistinguish under X-ray fluoroscopic guidance, it is difficult to alignX-ray image data with analysis image data, that is, X-ray image datawith ultrasound image data.

To identify an area specified in the ultrasonic diagnosis under X-rayfluoroscopic guidance, the units illustrated in FIG. 9 perform thefollowing processing in the first embodiment. FIG. 9 is a diagram ofprocessing units that carry out an image processing method performed bythe image processing system according to the first embodiment.

In the first embodiment, the aligning unit 156 b serving as the firstaligning unit included in the ultrasonic diagnostic apparatus 100 alignsfirst three-dimensional medical image data with second three-dimensionalmedical image data, the first three-dimensional medical image data andthe second three-dimensional medical image data being obtained bycapturing a certain tissue of a subject P. The first three-dimensionalmedical image data is three-dimensional medical image data in whichmotion of the certain tissue is analyzed. Specifically, the firstthree-dimensional medical image data is three-dimensional ultrasoundimage data. The second three-dimensional medical image data isthree-dimensional medical image data visualizing a specific tissueidentifiable in X-ray image data. In other words, the aligning unit 156b serving as the first aligning unit aligns three-dimensional ultrasoundimage data obtained by capturing the certain tissue of the subject Pwith second three-dimensional medial image data, the secondthree-dimensional medical image data being three-dimensional medicalimage data obtained by capturing the certain tissue of the subject P,being a three-dimensional medical image data visualizing a specifictissue that is identifiable in X-ray image data, and being differentfrom the three-dimensional ultrasound image data. The certain tissuecorresponds to the heart. Specifically, the second three-dimensionalmedical image data corresponds to three-dimensional X-ray CT image dataor three-dimensional MRI image data. The second three-dimensionalmedical image data is three-dimensional contrast enhanced image data,for example, and is three-dimensional X-ray CT image data in which thecoronary artery and the coronary vein are imaged or three-dimensionalMRI image data in which the coronary artery and the coronary vein areimaged. The specific tissue described above is an identifiable tissue inX-ray image data. Specifically, the specific tissue described above isan identifiable tissue in X-ray contrast enhanced image data obtained bycontrast enhanced radiography of the heart serving as the certaintissue. The specific tissue is the coronary artery or the coronary vein,for example. The second three-dimensional medical image data visualizingthe specific tissue may be three-dimensional MRI image data in which abloodstream is labeled by non-contrast radiography besidesthree-dimensional contrast enhanced image data, for example. Thefollowing describes the case where the second three-dimensional medicalimage data is three-dimensional contrast enhanced image data.

Because the enhancement of a coronary artery is higher than that of acoronary vein, three-dimensional X-ray CT image data in which thecoronary artery is enhanced or three-dimensional MRI image data in whichthe coronary artery is enhanced are preferably used as thethree-dimensional contrast enhanced image data. The following describesthe case where three-dimensional X-ray CT image data in which thecoronary artery serving as the specific tissue is enhanced is used asthe three-dimensional contrast enhanced image data serving as the secondthree-dimensional medical image data.

The output unit 157 a included in the ultrasonic diagnostic apparatus100 outputs, as output data, data obtained by adding alignmentinformation to the first three-dimensional medical image data(three-dimensional ultrasound image data) and to the secondthree-dimensional medical image data (three-dimensional contrastenhanced image data). Alternatively, the output unit 157 a outputssynthetic data obtained by aligning and synthesizing the firstthree-dimensional medical image data (three-dimensional ultrasound imagedata) with the second three-dimensional medical image data(three-dimensional contrast enhanced image data) as output data.

The acquiring unit 221 a serving as the second aligning unit included inthe X-ray diagnostic apparatus 200 receives the output data. Theacquiring unit 221 a then aligns the second three-dimensional medicalimage data with one or a plurality of pieces of X-ray image data, theX-ray image data being obtained by capturing the certain tissue of thesubject P in one or a plurality of capturing directions, andcorresponding to the respective capturing directions. Alternatively, theacquiring unit 221 a serving as the second aligning unit aligns thesecond three-dimensional medical image data with a piece of X-ray imagedata obtained by capturing the certain tissue of the subject P in acapturing direction and corresponding to the capturing direction. Thedisplay unit 223 included in the X-ray diagnostic apparatus 200 displaysimage data obtained by aligning the first three-dimensional medicalimage data with the X-ray image data of the certain tissue based on analignment result of the aligning unit 156 b serving as the firstaligning unit and the alignment result of the acquiring unit 221 aserving as the second aligning unit.

Specifically, the acquiring unit 221 a acquires three-dimensionalposition information of the specific tissue in a three-dimensionalcapturing space of the X-ray image data based on an alignment result ofthe second three-dimensional medical image data with the one or theplurality of pieces of X-ray image data. Alternatively, the acquiringunit 221 a acquires three-dimensional position information of thespecific tissue in the three-dimensional capturing space of a piece ofX-ray image data based on the alignment result of the secondthree-dimensional medical image data and the piece of X-ray image data.

More specifically, under the control of the acquiring unit 221 a, thedisplay unit 223 displays a projection image obtained by projecting thespecific tissue on the one or the plurality of pieces of X-ray imagedata when the second three-dimensional medical image data is arranged inthe three-dimensional capturing space of the X-ray diagnostic apparatus200. The acquiring unit 221 a then acquires the three-dimensionalposition information based on an operation for associating a position ofthe projection image with a position corresponding to the specifictissue in the one or the plurality of pieces of X-ray image dataperformed by an operator who refers to the display unit 223.Alternatively, under the control of the acquiring unit 221 a, thedisplay unit 223 displays a projection image obtained by projecting thespecific tissue on a piece of X-ray image data when the secondthree-dimensional medical image data is arranged in thethree-dimensional capturing space of the X-ray diagnostic apparatus 200.The acquiring unit 221 a then acquires the three-dimensional positioninformation based on an operation for associating the position of theprojection image with the position corresponding to the specific tissuein the piece of X-ray image data performed by the operator who refers tothe display unit 223.

In other words, the acquiring unit 221 a performs the alignment byassociating the “two-dimensional specific tissue depicted in thetwo-dimensional X-ray image data” with the “two-dimensional specifictissue obtained by projecting the three-dimensional specific tissuedepicted in the second three-dimensional medical image data in thecapturing direction of the X-ray image data” at three or more points.This enables the acquiring unit 221 a serving as the second aligningunit to perform the alignment using a piece of X-ray image data obtainedby capturing the specific tissue in a capturing direction.

The display unit 223 displays image data obtained by aligning the firstthree-dimensional medical image data or analysis image data generated byanalyzing the first three-dimensional medical image data with the X-rayimage data of the certain tissue, based on the three-dimensionalposition information of the specific tissue and based on a relativepositional relation between the first three-dimensional medical imagedata and the second three-dimensional medical image data.

The one or the plurality of pieces of X-ray image data on whichalignment is performed by the acquiring unit 221 a serving as the secondaligning unit are one or a plurality of pieces of X-ray contrastenhanced image data obtained by contrast enhanced radiography of thecertain tissue. Alternatively, the one or the plurality of pieces ofX-ray image data on which alignment is performed by the acquiring unit221 a serving as the second aligning unit are one or a plurality ofpieces of X-ray image data obtained by capturing the specific tissueinto which an instrument is inserted”. The instrument described above isa guide wire inserted into the coronary artery or the coronary vein, forexample. Because the guide wire is radiopaque, X-ray image data capturedwhen the guide wire is inserted depicts a region corresponding to thecoronary artery or the coronary vein clearly without injecting acontrast medium.

The following describes the case where the acquiring unit 221 a performsalignment using “a plurality of pieces of X-ray contrast enhanced imagedata obtained by contrast enhanced radiography in a plurality ofcapturing directions” as the “plurality of pieces of X-ray image data”.The contents described below are also applicable to the case where the“plurality of pieces of X-ray image data” correspond to “a plurality ofpieces of X-ray image data captured in a plurality of capturingdirections when the guide wire is inserted”. Still alternatively, thecontents described below are also applicable to the case where “a pieceof X-ray image data obtained by contrast enhanced radiography in acapturing direction” or “a piece of X-ray image data captured in acapturing direction when the guide wire is inserted” is used as “a pieceof X-ray image data”.

The acquiring unit 221 a, for example, receives output data and alignsthree-dimensional contrast enhanced image data with a plurality ofpieces of X-ray contrast enhanced image data obtained by capturing theheart of the subject P in a plurality of directions. Thus, the acquiringunit 221 a acquires the three-dimensional position information of thespecific tissue in the three-dimensional capturing space of theplurality of pieces of X-ray contrast enhanced image data.

Based on the three-dimensional position information of the specifictissue and the relative positional relation between thethree-dimensional ultrasound image data and the three-dimensionalcontrast enhanced image data, the display unit 223, for example,displays image data obtained by aligning the analysis image data(three-dimensional analysis image data) with the X-ray image data of thecertain tissue.

The following describes an example of the processing performed by theunits illustrated in FIG. 9. FIG. 10 and FIG. 11 are views forexplaining an example of the processing performed by the ultrasonicdiagnostic apparatus according to the first embodiment.

As described above, the aligning unit 156 b can align three-dimensionalcontrast enhanced region data segmented from three-dimensional contrastenhanced image data with three-dimensional analysis image data using theposition detecting system formed of the position sensor 160 and thetransmitter 161. In the present embodiment, for example, the aligningunit 156 b performs the alignment on three-dimensional end-diastolicanalysis image data (refer to FIG. 6) among a three-dimensional analysisimage data group generated from the three-dimensional ultrasound imagedata group.

The aligning unit 156 b performs the alignment on three-dimensionalcontrast enhanced region data (refer to the right figure in FIG. 10)obtained by extracting the coronary artery from the three-dimensionalX-ray CT image data (refer to the left figure in FIG. 10) by performingthreshold processing of the voxel value or a region expansion method. Inthe case where a time-series three-dimensional X-ray CT image data groupis acquired, the aligning unit 156 b performs the alignment onthree-dimensional contrast enhanced region data obtained by extractingthe coronary artery from the three-dimensional end-diastolic analysisimage data. The present embodiment may also use three-dimensionalcontrast enhanced region data segmented by the X-ray CT apparatus 300and the image processing apparatus 500, for example.

In the first embodiment, the aligning unit 156 b may perform thealignment without using the position detecting system. The aligning unit156 b, for example, adjusts the position and the angles of the threeaxes in the three-dimensional space so as to align projection imagesobtained by projecting three-dimensional ultrasound image data at apredetermined time phase in a plurality of viewpoint directions withrespective projection images obtained by projecting three-dimensionalX-ray CT image data at the predetermined time phase in a plurality ofviewpoint directions. Thus, the aligning unit 156 b aligns thethree-dimensional ultrasound image data with the three-dimensional X-rayCT image data at the same time phase. With this processing, the aligningunit 156 b can align the three-dimensional analysis data with thethree-dimensional contrast enhanced region data at the same time phase.

The output unit 157 a outputs, as output data, data obtained by adding“alignment information” to the analysis image data that is an analysisresult of the three-dimensional ultrasound image data” and to the secondthree-dimensional medical image data. The second three-dimensionalmedical image data may be “three-dimensional visualized region imagedata” obtained by extracting the “visualized specific tissue” from thesecond three-dimensional medical image data. Specifically, the outputunit 157 a transmits the three-dimensional analysis data and thethree-dimensional contrast enhanced region data at the same time phaseand the alignment information to the acquiring unit 221 a as the outputdata. The acquiring unit 221 a uses the alignment information, therebyarranging the three-dimensional analysis data and the three-dimensionalcontrast enhanced region data in the three-dimensional space in a manneraligned with each other as illustrated in FIG. 11.

Alternatively, the output unit 157 a outputs, as output data, syntheticdata obtained by aligning and synthesizing the analysis image data withthe second three-dimensional medical image data. The secondthree-dimensional medical image data may be three-dimensional visualizedregion image data obtained by extracting the “visualized specifictissue” from the second three-dimensional medical image data.Specifically, the output unit 157 a outputs synthetic data obtained byaligning and synthesizing the three-dimensional analysis data with thethree-dimensional contrast enhanced region data at the same phase asoutput data. The synthetic data is data illustrated in FIG. 11. Theoutput unit 157 a configures, when outputting the synthetic data as theoutput data, the synthetic data having specific information beingcapable of switching of display and non-display of the“three-dimensional ultrasound image data (three-dimensional analysisdata)” serving as the first three-dimensional medical image data and thethree-dimensional contrast enhanced image data (three-dimensionalcontrast enhanced region data) serving as the second three-dimensionalmedical image data and the specific information being separable.

Specifically, the “three-dimensional contrast enhanced image data(three-dimensional contrast enhanced region data)” of the synthetic datais used for processing performed by the acquiring unit 221 a. The“three-dimensional ultrasound image data (three-dimensional analysisdata)” is eventually displayed by the display unit 223. Thus, it ispreferable that display and non-display of these two pieces of data beswitchable and that the pieces of data be separable. The output unit 157a uses the brightness value as the specific information, for example.The output unit 157 a forms the three-dimensional analysis data as datarepresented by a brightness value of 511 gray-scale out of 512gray-scale and forms the three-dimensional contrast enhanced region dataas data represented by a brightness value of one gray-scale out of the512 gray-scale, thereby generating the synthetic data.

The present embodiment may use the three-dimensional contrast enhancedimage data as the output data. In this case, the image processing unit226 included in the X-ray diagnostic apparatus 200 extracts thethree-dimensional contrast enhanced region data from thethree-dimensional contrast enhanced image data.

The acquiring unit 221 a of the X-ray diagnostic apparatus 200 receivesthe output data. The acquiring unit 221 a uses the output data, therebyaligning the X-ray contrast enhanced image data with the ultrasoundimage data. FIG. 12 and FIG. 13 are views for explaining an example ofthe processing performed by the X-ray diagnostic apparatus according tothe first embodiment.

Under the control of the acquiring unit 221 a, the X-ray diagnosticapparatus 200 performs contrast enhanced radiography on the heart of thesubject P in a plurality of directions, thereby generating a pluralityof pieces of X-ray contrast enhanced image data. Under the control ofthe acquiring unit 221 a, the X-ray tube 212 irradiates the subject Pwith an X-ray in a first direction, and the X-ray detector 216 detectsthe X-ray passing through the subject P in the first direction asillustrated in FIG. 12, for example. Thus, the image data generatingunit 224 generates X-ray contrast enhanced image data in the firstdirection. Furthermore, under the control of the acquiring unit 221 a,the X-ray tube 212 irradiates the subject P with an X-ray in a seconddirection, and the X-ray detector 216 detects the X-ray passing throughthe subject P in the second direction as illustrated in FIG. 12, forexample. Thus, the image data generating unit 224 generates X-raycontrast enhanced image data in the second direction.

The acquiring unit 221 a uses the X-ray contrast enhanced image data inthe first direction, the X-ray contrast enhanced image data in thesecond direction, and the output data, thereby acquiring thethree-dimensional position information of the specific tissue. Becausethe specific tissue is the coronary artery, the X-ray contrast enhancedimage data in the first direction and the X-ray contrast enhanced imagedata in the second direction are X-ray contrast enhanced image data atan arterial phase. If the specific tissue is the coronary vein, theX-ray image data in the first direction and the X-ray image data in thesecond direction are X-ray contrast enhanced image data at a venousphase.

As illustrated in FIG. 13, the acquiring unit 221 a associates thecoronary artery depicted in the three-dimensional contrast enhancedregion data with the coronary arteries depicted in the respective piecesof X-ray contrast enhanced image data in the two directions. Thus, theacquiring unit 221 a acquires the three-dimensional position informationof the coronary artery in the three-dimensional contrast enhanced regiondata. The association is performed along the running path of thecoronary artery at three or more points.

The acquiring unit 221 a arranges the three-dimensional contrastenhanced region data in the three-dimensional capturing space of theX-ray diagnostic apparatus 200. The position at which thethree-dimensional contrast enhanced region data is arranged is set bythe operator, for example. Alternatively, the arrangement position is apreset position, for example. The acquiring unit 221 a causes the imageprocessing unit 226 to generate a projection image by projecting thespecific tissue (coronary artery) in the three-dimensional contrastenhanced region data on a plurality of pieces of X-ray image data. Underthe control of the acquiring unit 221 a, the image processing unit 226generates a projection image by projecting the three-dimensionalcontrast enhanced region data arranged in the three-dimensionalcapturing space in the first direction and the second direction, forexample.

Under the control of the acquiring unit 221 a, the display unit 223displays the projection image obtained by projecting thethree-dimensional contrast enhanced region data on the plurality ofpieces of X-ray contrast enhanced image data. The acquiring unit 221 aacquires the three-dimensional position information based on theoperation for associating the position of the projection image with theposition corresponding to the specific tissue in the plurality of piecesof X-ray contrast enhanced image data performed by the operator whorefers to the display unit 223. The operator, for example, performs amoving operation (associating operation) such that the projection imageof the coronary artery coincides with the coronary arteries viewed inthe respective pieces of X-ray contrast enhanced image data.

The operator performs the moving operation such that the projectionimage substantially coincides with the coronary artery depicted in theX-ray image data. The acquiring unit 221 a performs translation androtation of the three-dimensional contrast enhanced region data arrangedin the three-dimensional capturing space based on the amount of movementand the direction of movement of the projection image. The acquiringunit 221 a then acquires the position of the three-dimensional contrastenhanced region data subjected to the processing as thethree-dimensional position information. Based on the three-dimensionalposition information and the relative positional relation between thethree-dimensional contrast enhanced region data and thethree-dimensional analysis image data, the acquiring unit 221 arearranges the three-dimensional analysis image data in thethree-dimensional capturing space. The moving operation may possiblyexpand, reduce, or deform the projection image. In this case, thethree-dimensional contrast enhanced region data is expanded, reduced, ordeformed in the three-dimensional capturing space. In this case, thethree-dimensional analysis image data is expanded, reduced, or deformedafter being rearranged in the three-dimensional capturing space by theacquiring unit 221 a.

The image processing unit 226 projects the three-dimensional analysisimage data rearranged in the three-dimensional capturing space based onthe three-dimensional position information or the three-dimensionalanalysis image data rearranged in the three-dimensional capturing spacebased on the three-dimensional position information and then “expanded,reduced, or deformed” on the X-ray contrast enhanced image data of theheart of the subject P being captured in real time in a directiondesired by the doctor. In other words, the image processing unit 226generates image data by superimposing the projection image of thethree-dimensional analysis image data aligned in the three-dimensionalcapturing space on the X-ray contrast enhanced image data of the heart.The direction desired by the doctor is a direction for capturing X-raycontrast enhanced image data suitable for placement of the electrode.The direction desired by the doctor can be arbitrarily changed during anoperation. The image processing unit 226 projects the three-dimensionalanalysis image data on the X-ray contrast enhanced image data of theheart of the subject P being captured in real time in the direction thuschanged.

FIG. 14 is a view of an example of image data displayed in the firstembodiment. By referring to image data X illustrated in FIG. 14, thedoctor can place the electrode into a vein closest to the asynchronousarea while checking the asynchronous area in the projection image of thethree-dimensional analysis image data. Because the projection image ofthe three-dimensional analysis image data is a superimposed image,display and non-display thereof can be switched in response to a requestfrom the operator. In the present embodiment, the projection target ofthe three-dimensional analysis image data to be superimposed on theX-ray image data may only be the asynchronous area. The opacity of theprojection image of the three-dimensional analysis image data to besuperimposed can be arbitrarily changed. The X-ray image data on whichthe projection image of the three-dimensional analysis image dataaligned in the three-dimensional capturing space is superimposed is notlimited to the X-ray contrast enhanced image data. The X-ray image dataon which the projection image of the three-dimensional analysis imagedata aligned in the three-dimensional capturing space is superimposedmay be X-ray image data captured in a direction desired by the doctorwithout injecting a contrast medium.

The first embodiment may output the output data includingthree-dimensional ultrasound image data serving as the firstthree-dimensional medical image data. In this case, the image datasuperimposed on the X-ray image data captured in the direction desiredby the doctor is image data based on the three-dimensional ultrasoundimage data. The image data based on the three-dimensional ultrasoundimage data is ultrasound image data having a plurality of short-axisplanes including a short-axis plane of the asynchronous area, forexample.

The following describes a flow of the processing of the image processingsystem 1 according to the first embodiment with reference to FIG. 15 andFIG. 16. FIG. 15 is a flowchart for explaining an example of theprocessing performed by the ultrasonic diagnostic apparatus according tothe first embodiment. FIG. 16 is a flowchart for explaining an exampleof the processing performed by the X-ray diagnostic apparatus accordingto the first embodiment. FIG. 15 illustrates an example of processingperformed after initial alignment of the two-dimensional ultrasoundimage data and the three-dimensional X-ray CT image data is completedusing a position detecting system.

As illustrated in FIG. 15, the ultrasonic diagnostic apparatus 100according to the first embodiment acquires a three-dimensionalultrasound image data group of the heart (Step S101). The analyzing unit156 a generates a three-dimensional analysis image data group (StepS102). The aligning unit 156 b aligns three-dimensional analysis imagedata with three-dimensional contrast enhanced region data at the sametime phase (Step S103).

The output unit 157 a, for example, generates synthetic data by aligningand synthesizing the three-dimensional analysis image data with thethree-dimensional contrast enhanced region data as output data andoutputs the output data (Step S104). The processing is then terminated.

As illustrated in FIG. 16, the acquiring unit 221 a included in theX-ray diagnostic apparatus 200 according to the first embodimentdetermines whether output data is received from the ultrasonicdiagnostic apparatus 100 (Step S201). If no output data is received (Noat Step S201), the acquiring unit 221 a waits until output data isreceived.

By contrast, if output data is received (Yes at Step S201), theacquiring unit 221 a controls each unit of the X-ray diagnosticapparatus 200 so as to generate a plurality of pieces of X-ray contrastenhanced image data in a plurality of directions (Step S202).Specifically, the X-ray diagnostic apparatus 200 captures the heart ofthe subject P at the arterial phase in a plurality of directions.

Under the control of the acquiring unit 221 a, the display unit 223projects the three-dimensional contrast enhanced region data on theplurality of pieces of X-ray contrast enhanced image data, therebydisplaying the data in a superimposed manner (Step S203). The acquiringunit 221 a determines whether an associating operation for associatingthe projection image with the coronary artery in the plurality of piecesof X-ray contrast enhanced image data is received from the operator(Step S204). If no associating operation is received (No at Step S204),the acquiring unit 221 a waits until an associating operation isreceived.

By contrast, if an associating operation is received (Yes at Step S204),the acquiring unit 221 a acquires the three-dimensional positioninformation of the coronary artery in the three-dimensional capturingspace based on the associating operation (Step S205). Under the controlof the acquiring unit 221 a, the display unit 223 displays image dataobtained by aligning the three-dimensional analysis image data with theX-ray contrast enhanced image data (Step S206). The processing is thenterminated.

As described above, the first embodiment aligns the three-dimensionalultrasound image data with the two-dimensional X-ray image data with thethree-dimensional X-ray CT image data (or the three-dimensional MRIimage data) intervening therebetween. Specifically, the first embodimentcan identify, in the three-dimensional X-ray CT image data, an areacorresponding to a scanning area in the three-dimensional ultrasoundimaged data with the position detecting system provided with theposition sensor 160. Furthermore, the first embodiment can align aplurality of pieces of volume data at a voxel level based on tissueinformation depicted in the two areas.

As a result, the first embodiment can facilitate alignment of thethree-dimensional analysis image data based on the ultrasound image datawith the three-dimensional contrast enhanced region data. Because acoronary artery has a characteristic form, the first embodiment canfurther facilitate alignment of the three-dimensional contrast enhancedregion data at the arterial phase with the X-ray contrast enhanced imagedata at the arterial phase. In other words, the first embodiment canalign the ultrasound image data (three-dimensional analysis image data)with the X-ray contrast enhanced image data. Thus, the first embodimentmakes it possible to identify the area specified in the ultrasonicdiagnosis under X-ray fluoroscopic guidance. In the first embodiment,the doctor can place the electrode near the asynchronous area whilereferring to the projection image of the three-dimensional analysisimage data that can be displayed in a superimposed manner by thealignment.

Second Embodiment

The first embodiment describes the case where the three-dimensionalposition information is acquired based on an operation performed by theoperator. A second embodiment will describe the case wherethree-dimensional position information is automatically acquired with nooperation performed by the operator with reference to FIG. 17. FIG. 17is a view for explaining the second embodiment.

An image processing system 1 according to the second embodiment has thesame configuration as that of the image processing system 1 according tothe first embodiment explained with reference to FIG. 1. When performingthe alignment on the plurality of pieces of the X-ray image datacaptured in a plurality of capturing directions, an acquiring unit 221 aserving as a second aligning unit according to the second embodimentaligns the second three-dimensional medical image data withthree-dimensional X-ray image data reconstructed from the plurality ofpieces of X-ray image data by performing pattern matching between aplurality of pieces of three-dimensional image data. The acquiring unit221 a according to the second embodiment, for example, alignsthree-dimensional contrast enhanced image data with three-dimensionalX-ray contrast enhanced image data reconstructed from a plurality ofpieces of X-ray contrast enhanced image data by performing patternmatching between a plurality of pieces of three-dimensional image data.Examples of the pattern matching include processing usingcross-correlation, autocorrelation, mutual information, standardizedmutual information, and the correlation ratio.

In the case where data included in output data is three-dimensionalcontrast enhanced region data extracted from the three-dimensionalcontrast enhanced image data, the target of the pattern matching is thethree-dimensional contrast enhanced region data. Under the control ofthe acquiring unit 221 a, for example, an image processing unit 226back-projects the plurality of pieces of X-ray contrast enhanced imagedata captured in a plurality of directions in the three-dimensionalcapturing space, thereby reconstructing the three-dimensional X-raycontrast enhanced image data. In the second embodiment, thethree-dimensional X-ray contrast enhanced image data is reconstructedfrom a plurality of pieces of X-ray contrast enhanced image datacaptured in two directions, three directions, or 50 directions.

To reduce the load in the pattern matching, the following processing ispreferably performed, for example. The acquiring unit 221 a serving asthe second aligning unit aligns a three-dimensional region of interestset in the second three-dimensional medical image data with athree-dimensional region of interest set in the three-dimensional X-rayimage data. The acquiring unit 221 a, for example, aligns thethree-dimensional region of interest set in the three-dimensionalcontrast enhanced image data (or the three-dimensional contrast enhancedregion data) with the three-dimensional region of interest set in thethree-dimensional X-ray contrast enhanced image data.

As illustrated in FIG. 17, for example, the operator sets athree-dimensional region of interest (ROI) in the three-dimensionalcontrast enhanced region data. As a result, the image processing unit226 extracts “volume data E”, which is three-dimensional contrastenhanced region data of the three-dimensional ROI, for example.Furthermore, the operator sets a two-dimensional ROI in two pieces ofX-ray contrast enhanced image data as illustrated in FIG. 17, therebysetting a three-dimensional ROI. As a result, the image processing unit226 reconstructs “volume data F”, which is three-dimensional X-raycontrast enhanced image data of the three-dimensional ROI. Thesethree-dimensional ROI may be automatically set by the acquiring unit 221a based on the brightness.

The acquiring unit 221 a performs pattern matching between the volumedata E and the volume data F, thereby performing alignment therebetween.Thus, the acquiring unit 221 a acquires the three-dimensional positioninformation of the specific tissue (e.g., the coronary artery). Theprocessing described above may be performed on three-dimensional X-rayimage data reconstructed from a plurality of pieces of X-ray image datacaptured in a plurality of directions when a guide wire is inserted, forexample. Because the subsequent processing is the same as that describedin the first embodiment, the explanation thereof will be omitted.

The following describes a flow of the processing of the image processingsystem 1 according to the second embodiment with reference to FIG. 18.FIG. 18 is a flowchart for explaining an example of processing performedby an X-ray diagnostic apparatus according to the second embodiment.Because processing performed by an ultrasonic diagnostic apparatus 100according to the second embodiment is the same as that described in thefirst embodiment, the explanation thereof will be omitted.

As illustrated in FIG. 18, the acquiring unit 221 a included in an X-raydiagnostic apparatus 200 according to the second embodiment determineswhether output data is received from the ultrasonic diagnostic apparatus100 (Step S301). If no output data is received (No at Step S301), theacquiring unit 221 a waits until output data is received.

By contrast, if output data is received (Yes at Step S301), theacquiring unit 221 a controls each unit of the X-ray diagnosticapparatus 200 so as to generate a plurality of pieces of X-ray contrastenhanced image data in a plurality of directions (Step S302).Specifically, the X-ray diagnostic apparatus 200 captures the heart ofthe subject P at the arterial phase in a plurality of directions.

The acquiring unit 221 a receives setting of a three-dimensional ROI(Step S303). The acquiring unit 221 a extracts three-dimensionalcontrast enhanced region data of the three-dimensional ROI andreconstructs three-dimensional X-ray contrast enhanced image data of thethree-dimensional ROI from the plurality of pieces of X-ray contrastenhanced image data (Step S304). The acquiring unit 221 a performspattern matching between the three-dimensional contrast enhanced regiondata of the three-dimensional ROI and the three-dimensional X-raycontrast enhanced image data of the three-dimensional ROI (Step S305).

The acquiring unit 221 a acquires the three-dimensional positioninformation of the coronary artery in the three-dimensional capturingspace (Step S306). Under the control of the acquiring unit 221 a, adisplay unit 223 displays image data obtained by aligning thethree-dimensional analysis image data with the X-ray contrast enhancedimage data (Step S307). The processing is then terminated.

As described above, the second embodiment can automatically acquire thethree-dimensional position information of the specific tissue. Thus, thesecond embodiment can further facilitate alignment of the ultrasoundimage data (three-dimensional analysis image data) with the X-raycontrast enhanced image data.

The processing of each unit described in the first and the secondembodiments may be performed by the X-ray CT apparatus 300 and the imageprocessing apparatus 500. A part or all of the generation of theanalysis image data, the alignment of the ultrasound image data with theX-ray CT image data, the output of the output data, and the acquisitionof the three-dimensional position information of the specific tissue maybe performed by the X-ray CT apparatus 300 and the image processingapparatus 500, for example. The superimposed image of the analysis imagedata and the X-ray image data thus aligned may be generated by the X-rayCT apparatus 300 and the image processing apparatus 500. In other words,a specific aspect of distribution and integration of processing unitsdescribed in the first and the second embodiments is not limited to thatillustrated in the drawings. All or a part of the processing units maybe functionally or physically distributed and integrated in arbitraryunits depending on various types of loads and usage.

The first and the second embodiments describe the case where the firstthree-dimensional medical image data is three-dimensional ultrasoundimage data and the second three-dimensional medical image data isthree-dimensional X-ray computed tomography (CT) image data orthree-dimensional magnetic resonance imaging (MRI) image data,visualizing the specific tissue. The contents described in the first andthe second embodiments are applicable to any case as long as the firstthree-dimensional medical image data is three-dimensional medical imagedata in which motion of the certain tissue can be analyzed and thesecond three-dimensional medical image data is three-dimensional medicalimage data visualizing the specific tissue. The contents described inthe first and the second embodiments, for example, are applicable to thecase where the first three-dimensional medical image data isthree-dimensional MRI image data captured at a time phase when thecontrast of the myocardium is enhanced and the second three-dimensionalmedical image data is three-dimensional X-ray CT image data captured ata time phase when the contrast of the coronary artery or the coronaryvein is enhanced. Alternatively, the contents described in the first andthe second embodiments, for example, are applicable to the case wherethe first three-dimensional medical image data is three-dimensionalX-ray CT image data captured at a time phase when the contrast of themyocardium is enhanced and the second three-dimensional medical imagedata is three-dimensional X-ray CT image data captured at a time phasewhen the contrast of the coronary vein is enhanced.

The image processing method described in the first and the secondembodiments can be performed by a computer, such as a personal computerand a workstation, executing an image processing program prepared inadvance. The image processing program may be distributed over a networksuch as the Internet. Furthermore, the image processing program may berecorded in a computer-readable recording medium, such as a hard disk, aflexible disk (FD), a compact disc read-only memory (CD-ROM), amagneto-optical disk (MO), and a digital versatile disk (DVD), andexecuted by a computer reading the image processing program from therecording medium.

As described above, the first and the second embodiments can identify anarea specified in an ultrasonic diagnosis under X-ray fluoroscopicguidance.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An image processing system comprising: a firstaligning unit that aligns first three-dimensional medical image datawith second three-dimensional medical image data, the firstthree-dimensional medical image data and the second three-dimensionalmedical image data being obtained by capturing a certain tissue of asubject; an output unit that outputs, as output data, data obtained byadding alignment information to the first three-dimensional medicalimage data and to the second three-dimensional medical image data orsynthetic data obtained by aligning and synthesizing the firstthree-dimensional medical image data with the second three-dimensionalmedical image data; a second aligning unit that receives the output dataand aligns the second three-dimensional medical image data with one or aplurality of pieces of X-ray image data, the X-ray image data beingobtained by capturing the certain tissue of the subject in one or aplurality of capturing directions, and corresponding to the respectivecapturing directions; and a display unit that displays image dataobtained by aligning the first three-dimensional medical image data withthe X-ray image data of the certain tissue based on an alignment resultof the first aligning unit and the second aligning unit.
 2. The imageprocessing system according to claim 1, wherein the firstthree-dimensional medical image data is three-dimensional medical imagedata in which motion of the certain tissue is analyzable and the secondthree-dimensional medical image data is three-dimensional medical imagedata visualizing a specific tissue identifiable in X-ray image data. 3.The image processing system according to claim 2, wherein the one or theplurality of pieces of X-ray image data on which alignment is performedby the second aligning unit are one or a plurality of pieces of X-raycontrast enhanced image data obtained by contrast enhanced radiographyof the certain tissue or one or a plurality of pieces of X-ray imagedata obtained by capturing the specific tissue into which an instrumentis inserted.
 4. The image processing system according to claim 3,wherein the second aligning unit acquires three-dimensional positioninformation of the specific tissue in a three-dimensional capturingspace of the X-ray image data based on an alignment result of the secondthree-dimensional medical image data with the one or the plurality ofpieces of X-ray image data, and the display unit displays image dataobtained by aligning the first three-dimensional medical image data oranalysis image data generated by analyzing the first three-dimensionalmedical image data with the X-ray image data of the certain tissue,based on the three-dimensional position information of the specifictissue and based on a relative positional relation between the firstthree-dimensional medical image data and the second three-dimensionalmedical image data.
 5. The image processing system according to claim 4,wherein the display unit displays a projection image obtained byprojecting the specific tissue on the one or the plurality of pieces ofX-ray image data when the second three-dimensional medical image data isarranged in the three-dimensional capturing space, and the secondaligning unit acquires the three-dimensional position information basedon an operation for associating a position of the projection image witha position corresponding to the specific tissue in the one or theplurality of pieces of X-ray image data performed by an operator whorefers to the display unit.
 6. The image processing system according toclaim 4, wherein, when performing the alignment on the plurality ofpieces of the X-ray image data, the second aligning unit aligns thesecond three-dimensional medical image data with three-dimensional X-rayimage data reconstructed from the plurality of pieces of X-ray imagedata by performing pattern matching between a plurality of pieces ofthree-dimensional image data.
 7. The image processing system accordingto claim 6, wherein the second aligning unit aligns a three-dimensionalregion of interest set in the second three-dimensional medical imagedata with a three-dimensional region of interest set in thethree-dimensional X-ray image data.
 8. The image processing systemaccording to claim 1, wherein, the output unit configures, whenoutputting the synthetic data as the output data, the synthetic data asdata having specific information being capable of switching of displayand non-display of the first three-dimensional medical image data andthe second three-dimensional medical image data and being separable. 9.The image processing system according to claim 1, wherein the outputunit outputs, as the output data, data obtained by adding alignmentinformation to analysis image data that is an analysis result of thefirst three-dimensional medical image data and to the secondthree-dimensional medical image data or synthetic data obtained byaligning and synthesizing the analysis image data with the secondthree-dimensional medical image data.
 10. The image processing systemaccording to claim 1, wherein the first three-dimensional medical imagedata is three-dimensional ultrasound image data and the secondthree-dimensional medical image data is three-dimensional X-ray computedtomography (CT) image data or three-dimensional magnetic resonanceimaging (MRI) image data.
 11. An X-ray diagnostic apparatus comprising:a second aligning unit that receives, as output data obtained based onan alignment result of a first aligning unit, data obtained by addingalignment information to first three-dimensional medical image data andto second three-dimensional medical image data, the firstthree-dimensional medical image data and the second three-dimensionalmedical image data obtained by capturing a certain tissue of a subjector synthetic data obtained by aligning and synthesizing the firstthree-dimensional medical image data with the second three-dimensionalmedical image data, and that aligns the second three-dimensional medicalimage data with one or a plurality of pieces of X-ray image dataobtained by capturing the certain tissue of the subject in one or aplurality of capturing directions, the X-ray image data corresponding tothe respective capturing directions; and a display unit that displaysimage data obtained by aligning the first three-dimensional medicalimage data with X-ray image data of the certain tissue, based on analignment result of the first aligning unit and the second aligningunit.
 12. An image processing method comprising: aligning, by a firstaligning unit, first three-dimensional medical image data with secondthree-dimensional medical image data, the first three-dimensionalmedical image data and the second three-dimensional medical image databeing obtained by capturing a certain tissue of a subject; outputting,by an output unit, as output data, data obtained by adding alignmentinformation to the first three-dimensional medical image data and to thesecond three-dimensional medical image data or synthetic data obtainedby aligning and synthesizing the first three-dimensional medical imagedata with the second three-dimensional medical image data; receiving, bya second aligning unit, the output data and aligning, by the secondaligning unit, the second three-dimensional medical image data with oneor a plurality of pieces of X-ray image data obtained by capturing thecertain tissue of the subject in one or a plurality of capturingdirections, the X-ray image data corresponding to the respectivecapturing directions; and displaying, by a display unit, image dataobtained by aligning the first three-dimensional medical image data withthe X-ray image data of the certain tissue based on an alignment resultof the first aligning unit and the second aligning unit.